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I. INTRODUCTION 



In a modern theoretical context, one generally expects nonzero neutrino masses and asso- 
ciated lepton mixing. Experimentally, there has been accumulating evidence for such masses 
and mixing. All solar neutrino experiments (Homestake, Kamiokande, SuperKamiokande, 
SAGE, and GALLEX) show a significant deficit in the neutrino fluxes coming from the Sun 
[1]. This deficit can be explained by oscillations of the z/ e 's into other weak eigenstate(s), 
with Am^, of the order 10~ 5 eV 2 for solutions involving the Mikheev-Smirnov-Wolfenstein 
(MSW) resonant matter oscillations [2,3] or of the order of 10~ 10 eV 2 for vacuum oscillations. 
Accounting for the data with vacuum oscillations (VO) requires almost maximal mixing. The 
MSW solutions include one for small mixing angle (SMA) and one with essentially maximal 
mixing (LMA). 

Another piece of evidence for neutrino oscillations is the atmospheric neutrino anomaly, 
observed by Kamiokande [4], 1MB [5], SuperKamiokande [6] with the highest statistics, and 
by Soudan [7] and MACRO [8]. This data can be fit by the inference of — > u x oscillations 
with Am 2 tm ~ 3.5 x 1CT 3 eV 2 and maximal mixing sin 2 26> atm = 1 [6]. The identification 
v x = v T is preferred over v x = u steri i e at about the 2.5a level [9], and the identification v x = v e 
is excluded by both the Superkamiokande data and the Chooz experiment [10,11]. 

In addition, the LSND experiment has reported observing — > v e and — > v e oscil- 
lations with Am 2 LSND ~ 0.1 — 1 eV 2 and a range of possible mixing angles, depending on 
Am 2 LSND [12]. This result is not confirmed, but also not completely ruled out, by a similar 
experiment, KARMEN [13]. 

There are currently intense efforts to confirm and extend the evidence for neutrino os- 
cillations in all of the various sectors - solar, atmospheric, and accelerator. Some of these 
experiments are running; these include the Sudbury Neutrino Observatory, SNO, and the 
K2K long baseline experiment between KEK and Kamioka. Others are in development and 
testing phases, such as BOONE, MINOS, the CERN - Gran Sasso program, KAMLAND, 
and Borexino [14]. Among the long baseline neutrino oscillation experiments, the approxi- 
mate distances are L ~ 250 km for K2K, 730 km for both MINOS, from Fermilab to Soudan 
and the proposed CERN-Gran Sasso experiments. The sensitivity of these experiments is 
projected to reach down roughly to the level Am 2 ~ 10~ 3 eV 2 . There is strong motivation for 
another generation of experiments with even higher sensitivity that can confirm the — > v T 
transition with the values of Am 2 tm and sin 2 29 atm reported so far and carry out further 
measurements of various neutrino oscillation channels. 

Recently, there has been considerable interest in the idea of a muon storage ring that 
would serve as a "neutrino factory", i.e., a source of quite high intensity, flavor-pure neutrino 
and antineutrino beams: u^+u e ip^+u e ) from stored // _ 's (/U + 's) respectively [15]- [20]. Given 
the very high intensities anticipated to be of order 10 20 and perhaps even 10 21 muon decays 
per year in various preliminary studies, one can envision neutrino oscillation experiments 
with quite long baselines of order several thousand km, with commensurate sensitivity to 
various neutrino oscillation channels. 

One of the appeals of the muon storage ring/neutrino factory is that one can measure 
several different neutrino oscillation transitions, using both the (P M ) and v e (z/ e ) from a 
fx~ beam. In this paragraph we assume a fx~ beam for definiteness (figures below are 
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shown for neutrinos from both stored /i + and ji~ beams). In addition to a high-statistics 
measurement of i/ M -> ^, as a disappearance test for the — > u T oscillation, one has also 
various other channels. Among these are v e — > 9^, for which the signal is a "wrong-sign 
muon", fi + , and v e — » v T , which, in about 18 % of its decays, would also yield a wrong-sign 
muon. The measurement of the muon charge would be possible with either a magnetized 
iron detector or a combination of a massive water Cerenkov detector followed by a muon 
spectrometer. With sufficient detector capabilities, one could also search for r appearance, 
as is envisioned by the ICANOE and OPERA experiments at Gran Sasso [21,22], although 
this requires neutrino energies E v > 20 GeV to avoid kinematic suppression of r production. 

An important effect that must be taken into account in such experiments concerns the 
matter-induced oscillations which neutrinos undergo along their flight path through the 
Earth from the source to the detector. Given the typical density of the earth, matter effects 
are important for the neutrino energy range E ~ O(10) GeV and Am 2 tm ~ 3 x 10~ 3 eV 2 
values relevant for the long baseline experiments, in particular, for the oscillation channels 
involving v e , as we shall show below. Matter effects can also be important for the neutrino 
energy range E ~ O(10) MeV and Am 2 ~ 10~ 5 eV 2 involved in MSW solutions to the 
solar neutrino problem. After the initial discussion of matter-induced resonant neutrino 
oscillations in [2] , an early study of these effects including three generations was carried out 
in [23] . The sensitivity of an atmospheric neutrino experiment to small Am 2 due to the long 
baselines and the necessity of taking into account matter effects was discussed e.g., in [24]. 
After Ref. [3], many analyses were performed in the 1980's of the effects of resonant neutrino 
oscillations on the solar neutrino flux, and matter effects in the Earth were studied, e.g., [25] 
and [26], which also discussed the effect on atmospheric neutrinos (see also the review [27]). 
Recent papers on matter effects relevant to atmospheric neutrinos include [28,29]. Early 
studies of matter effects on long baseline neutrino oscillation experiments were carried out 
in [30]. More recent analyses relevant to neutrino factories include [15,16], [31]- [35]. 

In this paper we shall present calculations of the matter effect for parameters relevant to 
possible long baseline neutrino experiments envisioned for the muon storage ring/neutrino 
factory. In particular, we compare the results obtained with constant density along the neu- 
trino path versus results obtained by incorporating the actual density profiles. We study 
the dependence of the oscillation signal on both E/Am^ tm and on the angles in the leptonic 
mixing matrix. We also comment on the influence of Am^, and CP violation on the oscil- 
lations. Some of our results were presented in Ref. [36]. Additional recent studies include 
[37]- [40]. 

In a hypothetical world in which there were only two neutrinos, and is T , the — > v T 
oscillations in matter would be the same as in vacuum, since both have the same forward 
scattering amplitude, via Z exchange, with matter. However, in the realistic case of three 
generations, because of the indirect involvement of v e due to a nonzero Ui 3 , and because of 
the fact that v e has a different forward scattering amplitude off of electrons, involving both 
Z and W exchange, there will be a matter-induced oscillation effect on — > v T (as well as 
other channels). 

We consider the usual three flavors of active neutrinos, with no light sterile (=electroweak- 
singlet) neutrinos. This is sufficient to describe the solar and atmospheric neutrino deficit. 
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If one were also to include the LSND experiment, then, to obtain a reasonable fit, one would 
be led to include light electroweak-singlet neutrinos. Since the LSND experiment has not so 
far been confirmed, we shall, while not prejudging the outcome of the BOONE experiment, 
not include this in our fit. We calculate oscillation probabilities in the full 3x3 mixing 
case and we study when Am^ can be relevant. In most cases there is only one mass scale 
relevant for long baseline neutrino oscillations, Am^ (m ~ few x 1CT 3 eV 2 and we work with 
the hierarchy 

Am 2 21 = Am 2 sol < Am 2 31 « Am\ 2 = Am 2 atm (1.1) 

In our work we take into account the actual profile of the Earth, as given by geophysical 
seismic data [41] and compare the results with those calculated using the approximation 
of average density along the path of the neutrino. Further, when only one mass squared 
difference is relevant, we present the oscillation probabilities as functions of E/Am 2 , where 
here and below, E = E u . This way of presenting the results is useful since, for a given L 
value, it shows the matter effect for a wide range of E and Am 2 and hence can serve as 
an input in the choice of optimal beam energy (along with other considerations such as the 
cross section dependence a ~ E and the beam divergence ~ (LE)~ 2 , which, together, favor 
higher values of E to achieve a high event rate). We study how these oscillation probabilities 
vary with the different input parameters and discuss the influence of the matter effects on 
the sensitivity to each of these parameters. 



II. THEORETICAL FRAMEWORK 

We first recall the form of the lepton mixing matrix. Let us denote the flavor vectors of 
SU(2) x U(l) nonsinglet neutrinos as v = (v e , v T ) and the vector of electroweak-singlet 
neutrinos as x — (Xi> ••> XnJ- The Dirac and Majorana neutrino mass terms can then be 
written compactly as 

-*-fr™(£rZ){Z) +h * (2 ' 1) 

where Ml is the 3x3 left-handed Majorana mass matrix, Mr is a n s x n s right-handed 
Majorana mass matrix, and Mp is the 3-row by n s -column Dirac mass matrix. In general, 
all of these are complex, and (M L ) T = M L , (M R ) T = M R . Without further theoretical 
input, the number n s of electroweak singlet neutrinos is not determined. For example, in 
the minimal SU(5) grand unified theory (GUT), n s = 0, while in SO(10), n s = 3. Within 
this theoretical context, since the terms xJaCxkR are electroweak singlets, the associated 
coefficients, which comprise the elements of the matrix Mr, would not be expected to be 
related to the electroweak symmetry breaking scale, but instead, would be expected to be 
much larger, plausibly of order the GUT scale. Furthermore, the left-handed Majorana mass 
terms can only arise via operators of dimension at least 5, such as 

° = TT E K,M k e jm + e im e jk ) \C T a lCC{ L \ fc m + h.c. (2.2) 



M x 



a,b 
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where = (i/< o ,4)i is the left-handed, / = 1/2, Y — — 1 lepton doublet with generation 
index a (a = 1, 2, or 3), where £ a = e,/i, r, for a = 1,2,3, denotes a generic mass 
scale characterizing the origin of this term, and <fi is the standard model Higgs or H u in the 
super symmetric standard model. Because (2.2) is a nonrenormalizable operator, the success 
of the standard model as a renormalizable field theory then implies that Mx is much larger 
than the scale of electroweak symmetry breaking, and, within a GUT context, Mx would 
be of order the GUT scale, as with M R . The terms arising from the vacuum expectation 
values of the Higgs doublets then make up the submatrix M^. The resultant diagonalization 
of the matrix in eq. (2.1) then naturally leads to a set of 3 light masses for the three known 
neutrinos, generically of order m u ~ m 2 D /Mn, and n s very large masses generically of order 
M/j, for the electroweak singlet neutrinos. This seesaw mechanism is very appealing, since it 
can provide a plausible explanation for why the known neutrinos are so light [42] . Although 
the full leptonic mixing matrix is (3+n s ) x (3+n s ) dimensional, the light and heavy neutrinos 
largely decouple from each other so that, to a high degree of accuracy, one can describe the 
linear combinations of the (3 + n s ) mass eigenstates that form (3 + n s ) weak eigenstates in 
a decoupled manner, using a simple 3x3 matrix U, which, to high accuracy, is unitary, for 
the known neutrinos. This is determined by the diagonalization of the effective 3x3 light 
neutrino mass matrix 

M v = -M D M^ X M T D (2.3) 

and an n s x n s matrix for the heavy neutrino sector, which matrix will not be used here. 
The lepton mixing matrix can then be written as the unitary matrix 



U = R 23 KR 13 K*R 12 K' = 



( C12C13 s 12 c 13 s 13 e~ t5 \ 

-S12C23 - Ci 2 S23Si 3 e J<5 C12C23 - s 12 s 23 s 13 e tS s 23 c 13 
\ S12S23 - c 12 c 23 s 13 e tS -c 12 s 23 - s 12 c 23 s 13 e tS c 23 c 13 ) 



K' (2.4) 



where i?^ is the rotation matrix in the ij subspace, c^- = cos#y, Sj 3 - = sin#y, K = 
diag(e~ lS , 1, 1) and K' involves further possible phases due to Majorana mass terms that 
will contribute here. 

In passing, we note that although this theoretical context is appealing, various modifica- 
tions are possible. For example, string theory generically involves certain moduli fields which 
are singlets under the standard model gauge group, have flat superpotentials, and hence are 
massless in perturbation theory down to the energy scale where supersymmetry is broken. 
The spinor component fields, modulinos, can act as electroweak-singlet neutrinos, and may 
well have masses much less than the GUT scale (e.g. [43]). Moreover, models with a low 
string scale << M P i anck and large compact dimensions (e.g., [44]) also have implications for 
neutrino phenomenology. Here we shall work within the conventional seesaw-type scenario 
because of its simplicity and success in accounting for the most striking known feature of 
neutrinos, namely the fact that they are so light compared with the other known fermions. 

For our later discussion it will be useful to record the formulas for the various relevant 
neutrino oscillation transitions. In the absence of any matter effect, the probability that a 
(relativistic) weak neutrino eigenstate u a becomes u h after propagating a distance L is 
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P(y a - v b ) = <U - 4 £ Re(K abti j) sin 2 (^^) + 



4E 

+ 4 £ sin(^) cos(^) (2.5) 

i>j=i 

where 

= U ai UuU* aj U bj (2.6) 

and 

Am 2 . = m(^) 2 - m{y 3 f (2.7) 

Recall that in vacuum, CPT invariance implies P[pb —> v a ) = P[y a — > v^) and hence, for 
b = a, P{p a v a ) = P(y a — > For the CP-transformed reaction V a — > z/f, and the 

T- reversed reaction i/^ — > i/ a , the transition probabilities are given by the right-hand side of 
(2.5) with the sign of the imaginary term reversed. (Below we shall assume CPT invariance, 
so that CP violation is equivalent to T violation.) For most sets of parameters, only one 
mass scale is relevant for the neutrino oscillations of interest here, namely 

Am 2 atm = Amj 2 (2.8) 

In this case, CP (T) violation effects are negligibly small, so that in vacuum 

Pipa -> h) = P{Va -> »b) (2.9) 
P{Vb -> Va) = P("a -> Ub) (2.10) 

In the absence of T violation, the second equality (2.10) would still hold in matter, but even 
in the absence of CP violation, the first equality (2.9) would not hold. With the hierarchy 
(1.1), the expressions for the specific oscillation transitions are 

P(u^u T )=A\U 33 \ 2 \U^sin\^^) 



AE 

sin 2 (20 23 ) cos 4 (M sin 2 (^|^) (2.11) 



r>( \ A \ TT \2\ TT \2 ■ 2f^ m atm-L 

P{ye ^ ^\x) — 4|c/i3| \U23\ Sin ( 



AE 



= sin 2 (2£ 13 ) sin 2 (0 23 ) sin 2 (^|^) (2.12) 



P[v e - v T ) = 4|^33| 2 |^i3| 2 sin 2 (^^) 
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sin 2 (2^ 13 )cos 2 (^3)sin 2 (^|^) 



(2.13) 



In neutrino oscillation searches using reactor antineutrinos, i.e. tests of v e — > v e , the 
two-species mixing hypothesis used to fit the data is 



P(u e - u e ) = 1 - sin 2 (2^ reactor )sin 2 ( A/i,;: L 



AE 



(2.14) 



where Am 2 eactor is the squared mass difference relevant for V e — > P x . In particular, in the 
upper range of values of Am 2 tm , since the transitions V e — > z/ M and u e — > P T contribute to P e 
disappearance, one has 



PU. 



, e ) = l-sin 2 (2Msin 2 (^|^ 



i.e., 6 re actoT = 0i3, and the Chooz reactor experiment yields the bound [10] 

sin 2 (2#i 3 ) < 0.10 



(2.15) 



(2.16) 



which is also consistent with conclusions from the SuperK data analysis [6]. 

Further, the quantity "sin 2 (2# atm )" often used to fit the data on atmospheric neutrinos 
with a simplified two-species mixing hypothesis, is, in the three-generation case, 

sm 2 (2e atm ) = sin 2 (2fl 23 ) cos 4 (0 13 ) (2.17) 

Hence for small 6>i 3 , as implied by (2.16), it follows that, to good accuracy, 9 atm = 6*2 3 . 

III. CALCULATION OF MATTER EFFECTS 



The evolution of the flavor eigenstates of neutrinos is given by 

i-^-u = (^-UM 2 U^ + v) v 
dx \2E 



where 



v — Uu„ 



(3.1) 



(3.2) 




(3.3) 



M 



m\ 





m\ 



\ 


m\) 



[V2G F N e \ 


V o oo; 



(3.4) 
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Here N e is the electron number density and we have ^/2GfN £ [eV]= 7.6 x 10 _14 F e p [g/cm 3 ]. 

The atmospheric neutrino data suggests almost maximal mixing in the 2 — 3 sector. 
However, a small but non-zero Si 3 is still allowed, and this produces the matter effect in 
the traversal of neutrinos through the Earth. We use the bound (2.16) on sin 2 (2#i3) here, 
consistent with both the Chooz experiment [10] and the atmospheric neutrino data [6]. 

If we assume that the solar neutrino deficiency is explained by the small mixing angle 
(SMA) MSW solution or by vacuum oscillations, with the hierarchy of eq. (1.1), it follows 
that, for the relevant energies E > 1 GeV and path-lengths L ~ 10 3 — 10 4 km, only one 
squared mass scale, Am 2 atm) is important for the oscillations and the three-species neutrino 
oscillations can be described in terms of this quantity, Am 2 tm , and the mixing parameters 
sin 2 (26 l 23), and sin 2 (20 13 ), with negligible dependence on sin 2 (2# 12 ) and 5. 

In order to write down the probabilities of oscillation for long-baseline and atmospheric 
neutrinos, it is convenient to transform to a new basis defined by (e.g. [29]) 



v 



R23V 



(3.5) 



The evolution of v is given by 



h = —kr 13 k*r 12 m 2 rUkrIk* + V 



2E 



(3.6) 



In the one mass-scale approximation, this can be reduced to 



/ 2^ 2 3 Am 2 2 + V2G F N e 



H ~ 







V ^s 13 c 13 Am 2 32 e i5 








2 E C 12^ m 21 



^s 13 c 13 Aml 2 e i<5> 



2£ C 13 / -^ m 32 



(3.7) 



It can be seen now that in the basis (u e , z/ M , u T ) the three-flavor evolution equation decouples 
and it is enough to treat the two-flavor case. We define S and P by 



and 







fu e \ 




(x) = S 




\Vt) 






P EE 


\s 13 \ 2 = i 


-\s 3 



(0) 



(3.8) 



(3.9) 



Transforming back to the flavor basis (v e , v T ), the probabilities of oscillation become 



P{Ve 

P{v» 



v, 



Vn 



l-P 

P{Vu - 



C 2zP 



= 1 



S 2?,P 



~ S 23 C 23 



4 3 P + 2s 2 23 c 2 23 [Re(S 22 S 33 ) 
[2 - P - 2Re(S 22 S 33 )} 



(3.10) 
(3.11) 
(3.12) 
(3.13) 
(3.14) 
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Note that for the mass hierarchy (1.1), the CP- violating phases disappear from the oscilla- 
tion probabilities. In this case what we need to solve is the evolution equation for a two-flavor 
neutrino system. By subtracting from the diagonal the quantity ^Am| + -^GeN £ , this 
can be written in the form 



with 



Am 2 C 

A(x) = ^fcos(29 13 )-^N e (x) (3.16) 



B 



(3.17) 



For our purposes, we recall that the Earth is composed of crust, mantle, liquid outer 
core, and solid inner core, together with additional sublayers in the mantle, with particularly 
strong changes in density between the lower mantle and outer core. The density profile of 
the Earth is shown in Fig.l. The densities of the different layers are given in Table 1 as 
function of the normalized radius x = R/Re, Re = 6371 km being the radius of the Earth. 
The core has average density p core = 11.83 g/cm 3 and electron fraction Y £tCore = 0.466, while 
the mantle has average density p man tie = 4.66 g/cm 3 and Y e ,mantu = 0.494. 



Radius [Km] 


Density [g/cm 3 ] 


- 1221.5 


13.0885 - 8.8381a; 2 


1221.5 - 3480.0 


12.5815 - 1.2638a; - 3.6426a; 2 - 5.5281a; 3 


3480.0 - 5701.0 


7.9565 - 6.4761a; + 5.5283a; 2 - 3.0807a; 3 


5701.0 - 5771.0 


5.3197- 1.4836a; 


5771.0 - 5971.0 


11.2494- 8.0298a; 


5971.0 - 6151.0 


7.1089 - 3.8045a; 


6151.0 - 6346.6 


2.6910 + 0.6924a; 


6346.6 - 6356.0 


2.900 


6356.0 - 6371.0 


2.600 



Table 1 Density Profile of the Earth 



Since, to very good accuracy, the Earth is spherically symmetric (Fig.l), the neutrino 
flight path is described only by the zenith angle 9 Z (or rj = n — 9 Z ). For 



Ri+i „ . „ R% 

—— < sin 7] < — 
R 1 R 



(3.18) 



the neutrinos pass through 2i + l layers in the Earth. The distances traveled by the neutrinos 
in each of these layers are 
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Li = Rcosr]- \]R\ -R 2 sin 2 r] (3.19) 
L k = L 2i+2 _ k = ^Rl^-R 2 sin 2 - yj R 2 k -R 2 sin 2 77 , 2 < k <i (3.20) 
L i+ i = 2^R 2 -R 2 sin 2 r7 (3.21) 

Studies have been done using the average density of the Earth along the neutrino path. 
In this case the evolution equation can be easily solved and the probability of oscillation is 
given by 

P{v a -> 1*) = sin 2 (2# m ) sin 2 (cjL) (3.22) 
where to = y/ A 2 + B 2 and 9 m is the effective mixing in matter given by 

sin 2 (2# m ) = , (3.23) 

sin 2 (2^) + (cos(2tf) - 



Just as was the case with the application of the MSW analysis to solar neutrinos, the key 
observation is that although the angle 9, which is essentially #13 here, is small, the vanishing 
of the term in parentheses in the denominator of (3.23) renders the effective mixing angle 
m = vr/4, thereby producing maximal mixing in matter. The important point is that, given 
the range of densities in the layers of the Earth, and the value of Am 2 tm ~ 3.5 x 1CT 3 eV 2 , 
this matter resonance occurs for neutrino energies of order O(10) GeV, in the range planned 
for long baseline neutrino oscillation experiments. Since this effect clearly depends on the 
sign of Am 2 , the measurement of matter effects can give information on this sign. When one 
takes account of the actual variable-density situation in the Earth, it is necessary to perform 
a numerical integration of the evolution equation, which we have done. We also go beyond 
the one mass-scale approximation and study the effect of Am^, and 9 12 on the oscillations. 
In this case we calculate the oscillation probabilities for nonzero values of all six oscillation 
parameters (three angles, one phase, and two mass square differences) and discuss when the 
simpler cases are very good approximations. 



IV. RESULTS AND DISCUSSION 

For long baseline experiments like K2K, MINOS, and CERN to Gran-Sasso, the neutrino 
flight path only goes through the upper mantle. The density in this region is practically 
constant, and the oscillation probabilities can easily be calculated. The matter effects are 
small, but possibly detectable for the longer baselines. We show in Fig.2 an example for 
P(z/ M -> v e ) relevant for MINOS or the CERN to Gran-Sasso experiments (if SMA or VO 
solve the solar problem). 

However, there are several motivations for very long baseline experiments, since, with 
sufficiently high-intensity sources, these can be sensitive to quite small values of Am 2 and 
since the matter effects, being larger, can amplify certain oscillations and can, in principle, 
be used to get information on the sign of Am 2 tm . Hence we concentrate here on these very 
long baseline experiments; for these, the neutrino flight path goes through several layers of 
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the Earth with different densities, including the lower mantle for some. We show results 
for the Fermilab to SLAC path length L ~ 2900 km and for L ~ 7330 km, the distance 
from Fermilab to Gran Sasso. Path lengths corresponding to the distance BNL to SLAC, 
L ~ 4500 km, and BNL to Gran Sasso, L ~ 6560 km, are also considered. We have also 
performed calculations for L ~ 9200 km, the Fermilab to SuperKamiokande path length. 

In Fig. 3 we compare the probabilities calculated with constant density along the neutrino 
path versus the results obtained by numerically integrating the evolution equation with the 
actual density profile of the Earth, as given by [41]. The results are almost the same for most 
of the parameter range. However, at given energies, as for example for the second maximum 
in Pivp, — > v e )i the correction to the probability is of the order of 20%. The results in Fig. 3 
are obtained for the L = 7330 km distance, for which the beam goes through all layers 
of the mantle. In [36] we gave a series of similar comparisons of oscillation probabilities 
calculated with the constant density approximation and with the actual density function. In 
the following, we only present results obtained with the actual density function in the Earth. 

If one assumes the LMA solution to the solar neutrino problem, then for the L = 2900 km 
baseline, both Am 2 tm and Am^, have to be considered. In Fig.4 we show P(f M — > u e ) and 
the survival probability Piy^ — > u^) as functions of energy for Am^ m = 3.5 x 10~ 3 eV 2 
and sin 2 2^23 = 1, as suggested by the atmospheric data, and sin 2 2^13 = 0.1, the maximum 
value allowed, and two different choices of Am^ and sin 2 29u- One choice corresponds to 
the LMA solution, with Am 2 sol = 5 x 10" 5 eV 2 and sin 2 26>i 2 = 0.8. For this LMA case, the 
choice of the CP violating phase S is relevant; here we take 5 = and compare with nonzero 
5 below. The other choice is for the VO solution, with Am 2 sol = 10 -10 eV 2 and sin 2 2#i 2 = 1. 
The SMA solution gives the same results as the VO solution. One sees that the terms 
involving Am^ can have non-negligible effects on the z/ M — > v e oscillation probability for 
this path-length, especially at lower energies. 

As noted earlier, in the one-mass scale approximation, there are no CP violation effects in 
these oscillations; however, when we take into account Am 2 oi , we also have to consider CP- 
violating effects. We present in Fig. 5 a comparison showing the results for the probabilities 
P(z/ M — >■ v e ) and P{y e — >• v^) for 5 = and S — ir/2. We consider L = 2900 km, sin 2 29 2 3 = 1, 
sin 2 2^13 = 0.1 and the LMA solution for the solar neutrino problem. We can see that the 
effects of the CP-violating phase are small. Note however that for non-zero CP-violation, 
P{ v p, —* Ve) 7^ P{y e — > v^). For no CP- violation, even with matter effects, there is no 
difference between these two probabilities. Since with a muon storage ring, by switching 
between pT and fi + beams, one could obtain both and v e beams, there is the possibility of 
searching for the CP (actually T) violating difference Pivp, — > v e ) — P[ye ^v)- In practice, 
however, it would be difficult to identify the e~ from v e , given that the /i~ stored beam that 
yields the initial also yields P e , which produce e + in the detector, and given that it would 
be quite difficult to measure the sign of the e ± in planned detectors. An alternate method, 
to measure the asymmetry 

D = P ( U e -> ty) ~ Pipe -> g/j f4 v 

P(Ve -> ^) + P^e -> P„) ^ ' } 

is, in principle, possible, although it is complicated by the fact, as noted above, that D is 
rendered nonzero by matter effects even in the absence of CP violation (see also [16,37]). 
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If the solar neutrino problem is solved by the SMA or vacuum oscillations, CP-violation 
effects are not observable in the experiments of interest here. Indeed, even for the LMA 
solution, the CP violation would be very hard to detect for path lengths larger than ~ 3000 
km because of matter effects. 

For the Fermilab to Gran Sasso distance L ~ 7330 km (or the BNL to Gran Sasso distance 
L ~ 6560 km), the Am^, corrections are negligible, so we can analyze the problem using 
fewer relevant parameters: Am^ m , 9 13 and #23. We calculate the oscillation probabilities 
in long baseline experiments as a function of E/Am 2 , rather than using a particular value 
for Am 2 or the energy. The relevant ranges are Am 2 ~ few x 10~ 3 eV 2 and energies E 
of the order of tens of GeV. This way of presenting the results can be useful in studying 
the optimization of the beam energy. We calculate the oscillation probabilities for different 
values of the mixing angles #13 and 6*23 allowed by the atmospheric neutrino data and the 
CHOOZ experiment. 

We consider both neutrinos and antineutrinos. The matter effects reverse sign in these 
two cases; for antineutrinos, V in (3.4) is replaced with (—V). This implies that if Am 2 
is positive (as considered here), one can get a resonant enhancement of the oscillations for 
neutrinos, while for antineutrinos the matter effects would suppress the oscillations. The 
situation would be reversed if Am 2 were negative. The fact that the matter effects are 
opposite in sign for neutrinos and antineutrinos is well illustrated in Fig.2, where both 
results are presented, together with the vacuum case. 

In order to study the effects at different distances, we show the same type of graphs for 
both L = 7330 km and L = 2900 km. For L = 2900 km, the probabilities can be expressed 
as functions of E / Am 2 atm only for the SMA and VO solutions to the solar neutrino problem. 
For LMA, small Am 2 ol and CP violation corrections are added, as shown in Fig. 4 and Fig. 5. 

We first study the survival probability of v^. If the beam went through vacuum, the 
probability of oscillation would be given by Fig. 6 for a wide range of allowed values of 
sin 2 (26> 13 ). In matter, this probability becomes sensitive to all oscillation parameters for 
longer baselines such as 7330 km. In order to illustrate this, we calculate the probability 
for sin 2 (2#i 3 ) = 0.1, 0.04, and 0.01, and sin 2 (2# 23 ) = 0.8 and sin 2 (2# 23 ) = 1. The results 
are presented in Fig.7 and Fig.8. Evidently, the matter effect increases as #13 increases (and 
vanishes if #i 3 = 0). While the shift in the positions of the maxima and minima, as functions 
of E/Am 2 are small, there is a considerable change in the maximum at E/Am 2 ~ 3 x 10 3 . 
This is of great interest, since the use of a typical neutrino energy of E ~ 10 GeV (somewhat 
less than the stored muon energy) would produce this value of E/Am 2 , given the central 
value Am 2 = Am 2 tm ~ 3.5 x 10~ 3 eV 2 reported by SuperKamiokande [6]. 

We also want to compare the solution in vacuum (Fig.6), with the solution in matter for 
neutrinos (Figs. 7,8) and antineutrinos (Fig. 9). For antineutrinos the survival probability 
is not sensitive to the value of #i 3 . One can again see the opposite effects of matter on 
neutrinos and antineutrinos. The difference in the results for different mixing angles makes 
it possible, in principle, to use this probability for relatively precise measurements of the 
oscillation parameters. Measuring separately the probability for and can be very 
useful in detecting the matter effects and using these to constrain the relevant mixings and 
squared mass difference. Clearly, if one could use two path lengths, as may be possible with 
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a neutrino factory, this would provide more information and constraints. 

The relative effects of matter can be especially dramatic in the oscillation probability 
P{y e — > Vfj), since these directly involve v e . Since the v e beam would arise from a stored 
fi + beam, and the P^s from the decays of the /x + 's would produce /i + 's in the detector, 
the signature for the v e — > oscillation would be wrong-sign muons. As noted above, 
planned detectors would be capable of searching for such wrong-sign muons. Since this 
is a sub-dominant channel, the oscillation effect is small. If the beam went through the 
vacuum, neither P{v e — > nor the charge conjugate, P{p e — > would be enhanced 
by the matter effect (see Fig. 10, showing Piv^ — > is e ), which is equal to P[y e — > v^) 
for the present situation where CP violation is negligible). Because of the matter effect 
however, this probability can be strongly enhanced, as is evident in Fig. 11 and Fig. 12. For 
L = 7330 km, the enhancement is largest for E/Am 2 ~ 2.5 x 10 3 GeV/eV 2 . This is close 
to the ratio that one would get for a neutrino energy of E ~ 10 GeV, given the indication 
from the data that Am^ (m = 3.5 x 10~ 3 eV 2 . For L = 2900 km, the largest enhancement 
is obtained for E/Am 2 a factor of 3 lower. We also show in this case the results for the 
baselines corresponding to possible BNL-SLAC and BNL-Gran Sasso distances; see Fig. 13. 
As is evident, the matter effect can amplify P{y e — > v^) and enable this transition to be 
measured with good accuracy, thereby yielding very important information on the oscillation 
parameters. This probability is quite sensitive to the value of $13, so one should be able to 
use it for a good determination of this angle. This physics capability motivates careful design 
studies to optimize the choice of L and E for this measurement. The sensitivity to Am 2 
is also quite strong, due to the pronounced peak given by the matter effect in the relevant 
region. Note that for antineutrinos, the oscillation is suppressed (Fig. 14), so an independent 
measurement of the two channels (z/ M — > v e and — > v e ) would be very valuable. 

The atmospheric neutrino data tells us that the dominant oscillation channel is actually 
V(i v t- Consequently, it would be very useful to measure Piy^ — > u r ); this would provide 
further confirmation of this oscillation and could also provide further information on Am 2 
and 6*23- In addition to the MINOS experiment [14], the ICANOE and OPERA detectors that 
will operate in the CERN to Gran Sasso neutrino beam envision r appearance capabilities 
[21,22]. Results for Piv^ — > v T ) are presented in Fig. 15, and Fig. 16 shows P(pp — > v T ). 
Next, we present P{y e — > u T ) in Fig. 17 and P{p e — > v T ) in Fig.17. These calculations show 
that matter effects are important and enhance oscillations of the neutrinos and suppress 
oscillations of antineutrinos in the relevant region of parameters. By combining results from 
different types of measurements, in different channels of oscillations, the allowed parameter 
space can be strongly constrained, leading to precise measurements of all mixings and Am 2 . 

For a baseline over 9000 km, as would be the case for an experiment from Fermilab 
to SuperKamiokande, the main features discussed above remain true. Matter effects are 
significant for the oscillation of neutrinos for Am 2 in the region suggested by the atmospheric 
data and energies of the order of 10 GeV . Due to the matter effects, oscillations probabilities 
become very sensitive to #13. Matter effects also improve the sensitivity to Am 2 . Since matter 
effects for antineutrinos are opposite to those for neutrinos, independent measurements of 
neutrino and antineutrino oscillations would give a precise measure of the matter effects 
and, consequently, of the parameters relevant to the oscillations. Due to the longer path 
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through the Earth with bigger density, the matter effects can become even more dramatic. 
However, the statistics of the experiment would be limited by the lower neutrino flux at 
larger distances, and a careful study is necessary in order to choose optimal values of L and 
E. 

V. SUMMARY 

To summarize, in planning for very long baseline neutrino oscillation experiments, it is 
important to take into account matter effects. These effects are significant for the range of 
neutrino energies E of order 10's of GeV that are planned for these experiments, given the 
density of the Earth and the value of Am^ m ~ 3 x 10~ 3 eV 2 indicated by current atmospheric 
neutrino data. We have performed a study of these including realistic density profiles in the 
earth. Matter effects can be useful in amplifying neutrino oscillation signals and helping one 
to obtain measurements of mixing parameters and the magnitude and sign of Am^ (m . 
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FIG. 1. Density profile of the Earth. 
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E[GeV]/Am 2 [eV 2 ] 

FIG. 2. P{yn — > v e ) and Piy^, — > P e ) in matter and in vacuum for L = 730 km. (Figure legend is 
understood to include both cases. ) Here sin 2 (26>i 3 ) = 0.1 and sin 2 (26» 23 ) = 1. 
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FIG. 6. Hypothetical plot of P(z^ -> v^) in vacuum for L = 7300 km, L = 2900 km with sin 2 (26> 2 3) = 1, 
for comparison with other plots incorporating matter effects. 
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FIG. 7. P(z/ M v^) for L = 7300 km with sin 2 (20i3) = 0.1, .04, .01 and sin 2 (26> 23 ) = 1, 0.8. 
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FIG. 8. Same as Fig. 7 for L = 2900 km. 
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FIG. 9. Piy^ -> v„) for L = 7300 km, L = 2900 km with sm 2 (26» 23 ) = 1,0.8 and sin 2 (26>i 3 ) = 0.1. 
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FIG. 12. Same as Fig. 11 for L = 2900 km. 
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IG. 13. P(V p -> v e ) for L = 4500 km, L = 6560 with sin 2 (26>i 3 ) = 0.1,0.04,0.01 and sin 2 (26»2 3 ) = 1. 
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FIG. 14. P(Df, -> D e ) for L = 7300 km and L = 2900 km with sin 2 (26>i 3 ) = 0.1, .04, .01 and sin 2 (20 2 3) = 1. 
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FIG. 15. -> i/ T ) for L = 7330 km with sin 2 (26'i3) = 0.1,0.04,0.01 and sm 2 (26» 23 ) = 1. 
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FIG. 16. P(v^ -> D T ) for L = 7330 km with sin 2 (2(9i3) = 0.1 and sin 2 (26»2 3 ) = 1, 0. 



sin 2 (26 13 )=.l 
Sin(29 13 )=.04 
Sin(26 13 )=.01 




0.05 
0.04 

H 

r o.o3 

> 

! 0.02 
1 

0.01 




10' 



I 1 


— sinf28 13 )=.l 




- sin(26 13 )=.04 




— sinf28 13 )=.01 


- r ~ i /'' m / 


/^X L=7330km 







10' 



E[GeV]/Am 2 [eV 2 ] 



E[GeV]/Am[eVl 

FIG. 17. P(y e -> ;y r ) and P(i/ e -> i? T ) for X = 7330 with sin 2 (26»i 3 ) = 0.1, 0.04, 0.01 and sm 2 (26> 23 ) = 1. 



24 



0.50 



0.40 



0.30 



0.20 



0.10 



0.00 



neutrinos 
vacuum 
antineutrinos 

L=7330km 
sin 2 (26 23 )=1 . 
sin 2 (2e 13 )=.1 




E[GeV]/Am 2 [eV 2 ] 



0.20 



0.10 



0.00 



neutrinos 

vacuum 

antineutrinos 

L=2900km 

sin 2 (29 23 )=1. 
sin 2 (29 13 )=.1 




E[GeV]/Am 2 [eV 2 ] 



